An Analysis of the Metric Structure of the WeightSpace of Feedforward Networks and its Application

We study symmetries of feedforward networks in terms of their corresponding groups. We nd that these groups naturally act on and partition weight space into disjunct domains. We derive an algorithm to generate representative weight vectors in a fundamental domain. The analysis of the metric structure of the fundamental domain leads to improved evaluation procedures of learning results, such as local error bars estimated using maximum-likelihood and bootstrap methods. It can be implemented eeciently even for large networks. We demonstrate the approach in the area of nonlinear time series modeling and prediction.